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ABSTRACT 


We study a teacher incentive policy in Washington State that awards a financial bonus to National Board certified teachers in high poverty schools. Using a regression 
discontinuity design, we find that the bonus policy increased the proportion of certified teachers in bonus-eligible schools by improving hiring, increasing certifi- 
cation rates of incumbent teachers, and reducing turnover. Depending on the method, we estimate that the proportion of NBCTs in treated schools increased by about 
four to eight percentage points over the first five years of eligibility. However, the improvement in certification rates corresponds to a change of about 0.2-0.3% of a 


standard deviation in teacher quality per year and we do not find evidence that the bonus resulted in detectible effects on student test achievement. 


1. Introduction 


Teacher quality is among the most variable school-based influences on 
student learning. Using data from a random assignment experiment, 
Nye, Konstantopoulos, and Hedges (2004) estimate that within-school 
differences in teacher quality explain about 12-14% of student achieve- 
ment gains in math and about 7% of achievement gains in reading at the 
elementary school level. The magnitude of these findings is consistent with 
a large number of observational studies (Aaronson, Barrow, & Sander, 
2007; Rivkin, Hanushek, & Kain, 2005). Moreover, more recent research 
has linked teacher assignments to non-academic and distant outcomes like 
school attendance, educational attainment, and earnings (Chetty, 
Friedman, & Rockoff, 2014; Gershenson, 2016; Jackson, 2016; Koedel, 
2008). However, as with other schooling resources, not all students have 
equitable access to high-quality teachers, whether measured by observable 
credentials or effects on student achievement (Clotfelter, Ladd, & Vigdor, 
2005; Goldhaber, Lavery, & Theobald, 2015; Lankford, Loeb, & Wyckoff, 
2002; Mansfield, 2015; Sass, Hannaway, Xu, Figlio, & Feng, 2012). 

In response to these discrepancies, policymakers have become in- 


effective teachers in high poverty and other hard-to-staff schools. In 2014, 
the U.S. Department of Education announced requirements for all states to 
develop plans for addressing inequities in the staffing of schools. These 
plans require states to measure inequities in the assignment of low income 
and minority students to effective teachers. By law, states are required to 
report on several observable teacher characteristics: experience, teacher 
qualifications (e.g., state certification), and in-field teaching credentials 
(Williams, Adrien, Murthy, & Pietryka, 2016). States must additionally 
submit plans for reducing observed inequities. A number of states proposed 
offering additional financial incentives, including salary bonuses, loan 
forgiveness, or signing bonuses, to attract teachers to low income or other 
high needs schools (Klein, 2015a,b,c; Williams et al., 2016). 

There is currently limited evidence on the extent to which such 
targeted teacher bonuses affect student achievement in hard-to-staff 
schools. A number of studies have found that targeted bonuses improve 
teacher retention (Clotfelter, Glennie, Ladd, & Vigdor, 2008; Springer, 
Swain, & Rodriguez, 2016). There is relatively little evidence, however, 
on whether additional compensation for teachers in low income schools 
improves either teacher hiring or students’ academic outcomes. A key 
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characteristics of teachers and their preparation typically explain little 
of the variation in teachers' value-added contributions to student 
achievement (Aaronson et al., 2007; Harris & Sass, 2011; Rivkin et al., 
2005). There is also some uncertainty about whether teachers are suf- 
ficiently responsive to financial incentives to work in high-needs 
schools to improve student learning outcomes 
(Protik, Glazerman, Bruch, & Teh, 2015). It therefore remains an open 
question whether compensation policies targeted on teacher credentials 
or direct measures of productivity can substantially improve the quality 
of instruction in high-needs schools. 

In this study, we assess an incentive policy in Washington State 
designed to increase the supply of effective teachers in high poverty 
schools. The Challenging Schools Bonus (CSB) awarded a $5,000 an- 
nual bonus to teachers who earned certification through the National 
Board for Professional Teaching Standards (NBPTS) and worked in 
schools with a high proportion of students qualifying for subsidized 
lunches.' The CSB provides a good test of the potential for targeted 
incentives based on teaching credentials as it is one of the few cre- 
dentials consistently linked to student achievement gains in the teacher 
effectiveness literature (Cavalluzzo et al., 2015; Clotfelter, Ladd, & 
Vigdor, 2007,2010; Cowan & Goldhaber, 2016; Goldhaber & Anthony, 
2007). As with other indicators of teacher quality, NBCTs are less likely 
to teach in high poverty schools (Elfers & Plecki, 2014; Goldhaber, 
Choi, & Cramer, 2007; Humphrey, Koppich, & Hough, 2005; Sass et al., 
2012). At least five other states offer additional compensation for Na- 
tional Board certified teachers (NBCTs) that is similar to the CSB and 
two states have included National Board certification status in their 
teacher equity reports (National Board for Professional Teaching 
Standards, 2015; Williams et al., 2016). 

We study the effects of the NBCT bonus policy in Washington using 
a regression discontinuity design based on the schoolwide eligibility 
rule. We find that eligibility for the additional compensation increased 
the number of NBCTs in high poverty schools by improving hiring, 
encouraging certification among incumbent teachers, and reducing 
turnover among Board certified teachers. Over the first six years of the 
program, we estimate that eligibility increased the proportion of NBCTs 
by about 0.7-1.6 percentage points per year. The largest effects operate 
through increases in the number of teachers earning professional cer- 
tification. However, to the best of our knowledge, this is also one of the 
first papers to document the recruitment effects of differentiated com- 
pensation policies in education.” Despite the evident improvements in 
teacher staffing, we do not find positive student achievement effects 
from the bonus policy. Based on estimates of the relative effectiveness 
of NBCTs at raising student achievement found in the literature, our 
estimated effects on school staffing imply annual improvements in 
student learning of less than 0.001 standard deviations per year of 
eligibility. Consistent with this prediction, our direct estimates of the 
effects of the bonus policy on student achievement are near zero and 
not statistically significant. 


1 The NBPTS offers a national, voluntary certification process consisting of a 
yearlong sequence of assessments. Successful completion of the certification 
process is meant to indicate that a teacher has accomplished a high level of 
practice akin to board certification in medicine. Nationally, about 3% of tea- 
chers have earned NBPTS certification (Exstrom, 2011). The NBPTS credential 
is also commonly used by states as a signal of teacher quality. A majority of 
states accept NBPTS certification as a means of satisfying state licensing re- 
quirements and provide some financial incentive for teachers to earn certifi- 
cation (Exstrom, 2011; National Board for Professional Teaching Standards, 
2015). 

? We describe a few studies that examine recruitment below; however, each 
of these papers examines the effects of signing bonuses rather than policies that 
award permanent, but generally smaller, salary increases. 
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2. Background 


In this paper, we explore the effects of a policy awarding additional 
compensation to teachers in high poverty schools. These differentiated 
compensation policies may counteract two features of the public 
schooling system in the United States. First, a significant proportion of 
school financing comes from local taxes. Wealthier school districts tend 
to raise more revenue for public schools and offer higher salaries to 
teachers (Ushomirsky & Williams, 2015). Second, within school dis- 
tricts, teacher salaries have traditionally followed a uniform schedule 
with limited differentiation in pay. If low income schools offer fewer 
nonpecuniary amenities and equivalent compensation as wealthier 
schools, teachers may be less inclined to accept job offers 
(Rosen, 1986). Indeed, prior research has found that teachers in low 
income schools, on average, are less effective and less likely to be fully 
qualified to teach (Goldhaber et al., 2015; Lankford et al., 2002; Sass 
et al., 2012). Differentiated compensation policies should therefore 
affect attrition rates and the composition of the applicant pool in tar- 
geted schools. In addition, the policy we focus on, which incentivizes a 
particular form of professional certification, may additionally affect the 
certification and labor supply decisions of incumbent, uncertified tea- 
chers. 

By increasing the value of a teaching position in hard-to-staff 
schools relative to other teaching positions, targeted financial in- 
centives should improve the retention of teachers in such schools. 
Research on the relationship between district salaries and mobility 
supports this hypothesis (Hanushek, Kain, & Rivkin, 2004; Hendricks, 
2014; Imazeki, 2005; Lankford et al., 2002). More relevant to the 
program we study, Clotfelter et al. (2008) found that a temporary North 
Carolina program that awarded $1,800 bonuses to math, science, and 
special education teachers in high poverty or low achieving schools 
reduced the turnover of targeted teachers by about 17%. 
Springer et al. (2016) assessed a pilot Tennessee program that paid 
$5,000 bonuses to highly-rated teachers in low-achieving schools and 
found that receipt of the bonus improved retention among teachers in 
tested grades and subjects. These findings suggest that differentiated 
compensation policies can affect the composition of teachers by altering 
who leaves high poverty schools. 

A similar argument suggests that targeted incentives should im- 
prove the recruitment of new teachers into hard-to-staff schools. Better 
compensation may increase the likelihood that effective mobile tea- 
chers take positions in high-needs schools. Although there is little evi- 
dence about the effects of differentiated compensation policies on tea- 
cher recruitment, two studies of temporary recruitment bonuses have 
found that such policies increase the likelihood that teachers take po- 
sitions in eligible schools. Glazerman, Protik, Teh, Bruch, and 
Max (2013) analyzed a policy that provided a randomized group of 
schools the opportunity to offer high-performing teachers $20,000 bo- 
nuses to transfer to a low-achieving school for at least two years. They 
found clear evidence of recruitment effects: teachers recruited to eli- 
gible positions were 43 percentage points less likely to have less than 
six years of experience and more than twice as likely to possess National 
Board certification. Steele, Murnane, and Willett (2010) studied a 
policy initiative in California, the Governor's Teaching Fellowship, that 
conditioned a $20,000 scholarship on teaching in a low-performing 
school for four years. As with the findings from the Teacher Transfer 
Initiative, they found a substantial increase in the likelihood that tar- 
geted teachers work in such schools. There is also complementary 
evidence from similar recruitment programs in medicine (Pathman, 
Konrad, King, Taylor, & Koch, 2004; Rabinowitz, Diamond, Markham, 
& Hazelwood, 1999) and law (Field, 2008). 

Finally, beyond altering the incentives for already-certified tea- 
chers, incentive policies like the CSB also increase the value of ob- 
taining particular credentials. State policy or teacher salary contracts 
frequently provide additional compensation to any teachers holding 
academic degrees, particular subject-area endorsements, or National 
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Board certification. Although there is little evidence on how such po- 
licies affect the human capital investments or professional certification 
decisions of teachers, Feng and Sass (2017) have found that tuition 
reimbursement policies increased the likelihood that teachers in Florida 
earned endorsements in shortage areas. The implications of increased 
certification rates for high needs schools targeted by differentiated 
compensation policies, however, are ambiguous. Increasing profes- 
sional certification among incumbent teachers may improve instruc- 
tional effectiveness if they process of obtaining certification improves 
teaching practice. There is little evidence from student achievement 
data that participating in the NBPTS certification process improves 
practice (Chingos & Peterson, 2011; Goldhaber & Anthony, 2007; 
Harris & Sass, 2009), although Sato, Wei, and _ Darling- 
Hammond (2008) found that participation does improve teachers’ 
ability to perform the sorts of tasks that NBPTS assesses. On the other 
hand, teachers’ participation in the certification process, which NBPTS 
estimates requires 200-400 h, may disrupt student learning (Goldhaber 
& Anthony, 2007; Harris & Sass, 2009). Further, Goldhaber and 
Hansen (2009) have argued that earning professional certification 
provides other potential employers with an observable signal of a tea- 
chers’ effectiveness and increases the probability that effective teachers 
switch schools. 


2.1. Washington's challenging schools bonus program 


Washington State has awarded a salary incentive for NBCTs since 
the 1999-2000 school year. Initially set at 15% of salary, the state fixed 
the bonus at $3,500 per year in 2000 and raised it to $5,000 in 2007. At 
the same time, Washington introduced an additional bonus for teachers 
in high poverty schools. The program, called the Challenging Schools 
Bonus (CSB), awards an additional $5,000 to NBCTs.° Following the 
increase in the standard bonus and the introduction of the CSB, the 
number of NBCTs in Washington rose substantially. During the first 
year of the new bonus programs, the number of new teachers earning 
certification increased from 489 to 922 (Plecki et al., 2010). By 2014, 
Washington was producing the most new NBCTs nationwide 
(National Board for Professional Teaching Standards, 2014a). Over the 
same time period, annual state spending on bonuses for NBCTs in- 
creased from $10,000,000 to $45,000,000 (Office of the Superintendent 
of Public Instruction, 2014). 

The program targets teachers in the state's highest poverty schools 
based on the share of students eligible for free- or reduced-price lunch 
(FRL) programs. The minimum enrollment share, listed in Table 1, 
varies across years and school level. Initially, OSPI used two sources of 
data on FRL enrollment to determine school eligibility. The first of these 
is based on information reported in the state's administrative data 
system (Core Student Record System, CSRS).* Districts also report 
counts of school enrollment and FRL participation to the state's Child 
Nutrition Services (CNS). During the first year of the program, schools 
with FRL enrollment exceeding 70% of total enrollment were eligible 
for the bonus. Thereafter, eligible schools were elementary schools with 
FRL enrollment exceeding 70% of total enrollment, middle schools with 
FRL enrollment exceeding 60%, and high schools with FRL enrollment 
exceeding 50%. Until 2011, schools were also grandfathered in based 
on their FRL enrollment share in the previous year. The three thresholds 
are quite close to the 75th percentile of school FRL enrollment share by 
school level, so the program targets roughly the highest poverty 


3 The amount of the bonus is prorated by the proportion of time teachers 
spend in an eligible school. Starting with the 2011-2012 school year, the 
amount of the bonus was reduced to $3,000 for the first year a teacher earns 
National Board certification. 

4 The Comprehensive Education Data and Research System (CEDARS) re- 
placed CSRS in 2010. We refer to this as the “CSRS” measure throughout for 
simplicity. 
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Table 1 
Challenging schools bonus eligibility rules. 
Year Data Prior Year Threshold 
Source 
Elementary Middle High 
Schools Schools Schools 
2007-2008 CSRS/ 70% 70% 70% 
CNS 
2008-2009 CSRS/ Yes* 70% 60% 50% 
CNS 
2009-2010 CSRS Yes 70% 60% 50% 
2010-2011 CSRS Yes 70% 60% 50% 
2011-2012 CSRS No 70% 60% 50% 
2012-2013 CSRS No 70% 60% 50% 


Notes: Eligibility for Challenging Schools Bonus by school level and year. 
Elementary schools are defined as those with highest grade of 6th or lower. 
Middle schools are defined as those with a highest grade of 7th-9th grades. High 
schools are defined as those with a highest grade of 10th-12th. “Data source” 
denotes the data series used to estimate FRL enrollments. CSRS = Core Student 
Record System. In 2009-2010, the Comprehensive Education Data and 
Research System (CEDARS) replaced the CSRS; we maintain the labeling for 
simplicity. CNS = Child Nutrition Services report. “Prior year” indicates schools 
receiving the Challenging Schools Bonus in a prior year were grandfathered 
during the current school year. Schools serving fewer than 30 students are also 
excluded from eligibility unless they are the largest school at that grade level in 
the district. 

* To qualify for the Challenging Schools Bonus, school must have had 70% 
FRL enrollment in prior year regardless of grade level. 


quartile of schools in the state. 

The introduction of the incentive policies coincided with a major 
effort to increase the number of NBCTs in Washington State through an 
increase in the standard bonus for NBCTs, conditional loans for appli- 
cation fees, and candidate support and networking initiatives (Elfers & 
Plecki, 2014; National Board for Professional Teaching Standards, 
2010b). Consequently, the number of NBCTs in Washington State in- 
creased substantially in both CSB-eligible and CSB-ineligible schools 
shortly after the introduction of the policy. Nonetheless, the proportion 
of NBCTs in high poverty schools relative to low poverty schools in- 
creased over this time period. In 2007, 2.7% of teachers in low poverty 
schools (those schools never eligible for the bonus) and 2.0% of tea- 
chers in high poverty schools (those schools ever eligible) were NBCTs. 
By 2013, the gap in Board certification between low- and high poverty 
schools had reversed, with 9.9% and 11.3%, respectively, having 
earned certification. These trends are consistent with the findings of 
Elfers and Plecki (2014) and Simpkins (2011), who document that the 
proportion of NBCTs and candidates for Board certification in high 
poverty schools increased after 2007. 

To understand how the bonus policy is likely to affect teachers’ 
certification and labor supply decisions, it is helpful to understand the 
timing of the NBPTS process and CSB eligibility determination. The full 
process spans an approximately two year period (summarized in Fig. 1). 
OSPI typically determines eligibility for the upcoming school year in 
May or June based on enrollment data from the previous October. 
During the following school year, districts notify the state of eligible 
teachers working in their schools. Until the 2011-12 school year, Wa- 
shington disbursed funds to districts after receiving these reports. 
Currently, the state disburses the bonus funds to school districts in the 
July following the conclusion of the school year and requires districts to 
pay out the bonus by August 31. Given the timing of eligibility de- 
termination, most staffing outcomes in a particular school year are 
likely related to eligibility during the specified year. As shown in Fig. 1, 
the state announces bonus eligibility after the NBPTS application period 
closes for the upcoming school year (usually on December 31 during the 
years of this study). Although the NBPTS assessment cycle runs through 
June, teachers are unlikely to know their school's eligibility in time to 
apply for certification before their school becomes eligible. At the same 
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CSB School FRL Enrollment Eligibility 
Year t-/_ | Count Released 
AY tl Begins 
NBPTS Registration Assessment 
Window Closes Window Closes 
Month 8 9 10 11 #12 ~«21 2 3 4 5 6 7 
Certification Results 
NBPTS Released 
AYt School 
Year t Bonus Funds 
CSB Begins Released 


Fig. 1. Timeline of NBPTS certification and bonus processes. Source: American Federation of Teachers & National Education Association (2007, 2008) and 


National Board for Professional Teaching Standards (2010a). 


time, the announcement comes early enough that certified teachers 
may still participate in the job market for the upcoming school year. 
Teachers contemplating taking new positions during the spring or 
summer will generally know whether their current or potential schools 
will be eligible for the bonus during the coming school year. Newly 
certified teachers, who generally receive notification in the preceding 
December, should also be able to participate in the spring or summer 
job market. 


3. Data and empirical methods 
3.1. Data 


In this study, we use data on student assessments and teacher 
staffing from two databases maintained by the Office of the 
Superintendent of Public Instruction (OSPI). We construct data on 
teacher turnover and credentials using the S-275, which is the em- 
ployment reporting system for public schools in Washington State. 
Districts report school employees who have an employment contract in 
place by October 1 of each school year. Beyond school of employment, 
the S-275 includes information about employees’ assignments and 
teaching credentials. These include educational attainment, experience, 
salary, and teacher demographics. Using this database, we construct a 
panel of teachers for the school years 2001-2002 through 2012-2013. 
We define teachers as those with base contracts that have assignments 
in classroom teaching positions and assign teachers to a school if their 
reported time in that building exceeds 50% of their total full-time 
equivalency.” Using this assignment rule, we include all teachers 
working in Washington public schools that qualify to be considered for 
CSB eligibility.° The remaining sample represents about 90% of public 
school employees who are reported as working in classroom teaching 
positions. 

As we are primarily interested in how the bonus policy affects tea- 
cher retention, we define a teacher as moving if she is not assigned to 


° About 18% of all teaching staff reports a position in more than one building. 
However, this includes teachers with an assignment reported at the district 
office and may overstate the number of teachers who work in more than one 
school. 

© We omit schools that ever enroll fewer than 30 students, which are gen- 
erally not eligible to qualify for the CSB, and schools missing enrollment in- 
formation required for determination of eligibility. We also omit schools with 
no K-12 enrollment (2 schools) or teacher staffing information (17 schools) 
reported in any year. In addition, in the eligibility data provided by OSPI, there 
were three schools whose reported CSB status in one year did not correspond 
either to the reported FRL enrollment data or to eligibility reports from other 
years. Because we could not verify with OSPI whether these schools actually 
received the bonus, we dropped them from our study. The final sample includes 
1,698 schools, or 98.7% of the schools subject to the eligibility determination 
for the CSB. 


141 


the same school in the S-275 during the following academic year.” This 
includes teachers who switch schools and those who exit the public 
school system. Teachers earn the bonus if they do not leave their school 
at the end of the year it becomes eligible. Hence, we match the mobility 
data to a school's eligibility in the upcoming school year. 

We supplement the S-275 with data on National Board candidates 
provided by NBPTS. This data includes the year of submission and as- 
sessment results of all applications originating from public or private 
schools in Washington State. Our measure of National Board certifica- 
tion therefore only includes teachers who initially obtained certification 
in Washington and may understate the total number of NBCTs. To the 
extent that the bonus policy incentivizes cross-state migration to eli- 
gible schools, our estimates of staffing effects would likely be down- 
ward biased. However, cross-state teacher mobility appears to be re- 
latively rare. During the 2011-2012 school year, while nearly 7% of 
teachers had worked in a different public school in the same state 
during the previous year, only 0.5% of teachers had worked in a public 
school in another state (National Center for Education Statistics, 2015). 
There are additionally very few NBCTs in Oregon and Idaho, the two 
states that share a border with Washington (National Board for 
Professional Teaching Standards, 2014b). Given the relative rarity of 
both cross-state mobility and National Board certification, the degree of 
measurement error in our NBCT indicator is likely to be small. 

In most cases, we do not include individual level characteristics in 
our models. Individual teacher characteristics may be endogenous to 
bonus eligibility and therefore should be excluded. We therefore use 
data averaged at the school level in our analysis. We present summary 
statistics for teachers across all years of our sample in Table 2. Eligible 
schools constitute about 22% of the schools in our sample. In CSB 
schools, 14.1% of teachers depart each year, which is fairly similar to 
the statewide average of 12.9%. However, we do find some differences 
in observable teacher credentials: CSB teachers have about an average 
of about one year less teaching experience and are about 3 percentage 
points less likely to have an advanced degree. As a first indication that 
the CSB program may influence the supply of teachers, we find that 
about 8% of CSB teachers in our sample possess an NBPTS teaching 
certificate, compared to the sample average of 6%. 

The student-level data includes student demographic information, 
annual math and reading assessment data in grades 3-8 and 10, and 
information on participation in various special programs. Because our 
identification strategy ought to provide unbiased estimates of treatment 


7 The S-275 includes all teachers employed by October 1 of the reporting 
year, so our sample may exclude a small number of teachers who are laid off 
and subsequently recalled or who sign a contract after October 1 and stay at the 
school for only one year. Given that teachers are only eligible for National 
Board certification if they have more than 3 years of experience, this form of 
measurement error is unlikely to substantially affect our measurement of 
NBCTs. 
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Summary statistics. 
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Table 3 
Discontinuities in the density of the forcing variable. 


Full Sample CSB Schools 


Mean SD N Mean SD N 


Forcing Variable 


Eligibility Rule Current enrollment 


Panel A. Teacher characteristics 


NBCT 0.063 0.072 10,188 0.076 0.080 2,245 
NBCT candidate 0.018 0.034 10,188 0.024 0.043 2,245 
Salary 60,470.61 6,090.10 10,188 60,528.07 6,664.22 2,245 
Experience 13.485 2.777 10,188 12.506 2.862 2,245 
Leave school 0.129 0.082 10,188 0.141 0.094 2,245 
New to school 0.128 0.086 10,188 0.143 0.098 2,245 
Advanced 0.656 0.113 10,188 0.624 0.121 2,245 
degree 
Panel B. Student Characteristics 
Math test 0.010 0.397 28,701 —0.369 0.320 6,357 
Reading test 0.006 0.350 29,655 —0.372 0.282 6,686 
Asian/Pacific 0.087 0.095 29,664 0.081 0.113 6,689 
Islander 
Black 0.050 0.074 29,664 0.081 0.119 6,689 
Hispanic 0.178 0.198 29,664 0.430 0.292 6,689 
White 0.624 0.227 29,664 0.344 0.239 6,689 
Gifted 0.053 0.102 29,664 0.037 0.090 6,689 
Limited English 0.059 0.095 29,664 0.162 0.151 6,689 
proficiency 
Special 0.084 0.053 29,664 0.089 0.056 6,689 
education 
services 
Free/reduced- 0.448 0.240 29,664 0.784 0.139 6,689 


price lunch 


Notes: Summary statistics for teacher and student observations. Observations 
are at the school-year (for teachers) and school-grade-year level (for students). 
Summary statistics are for all observations between 2008 and 2013. 
Observations for the full analysis sample are displayed under “all schools” and 
for currently-eligible CSB schools under “CSB schools.” 


effects without relying on pre-test information, we use all years for 
which testing outcomes are available. For 2008-2013, we use stan- 
dardized reading assessments in grades 3-8 and 10 and standardized 
math assessments in grades 3-8. In 2010, the state replaced the 10th 
grade standardized math assessment with end-of-course assessments in 
algebra and geometry. Given that the decisions about the timing of end- 
of-course assessments may be endogenous to bonus eligibility, we only 
consider outcomes on the standardized 10th grade math assessment 
from 2008-2010. We standardize all assessments by grade and year. We 
present summary statistics for students in Panel B of Table 2. Un- 
surprisingly, CSB-eligible schools are quite different than the state as a 
whole. Student achievement is nearly 0.4 standard deviations lower in 
CSB schools than in other schools during the same time period. Students 
are also much more likely to be members of an underrepresented 
minority: 43% and 8% of students in CSB schools are Hispanic and 
African-American, respectively, compared to just 17% and 5% overall. 
Not surprisingly given these demographics, CSB students are nearly 
three times more likely to participate in bilingual programs than the 
state average. 


3.2. Research design 


To identify the effect of the bonus policy on student achievement 
and teacher staffing, we exploit the discontinuous relationship between 
school-wide FRL enrollment share and bonus eligibility using a re- 
gression discontinuity design (RDD). The RDD relies on the fact that 
schools with comparable poverty levels near the eligibility thresholds 
fall into different treatment states. By focusing on changes in outcomes 
at the eligibility threshold, the regression discontinuity approach ig- 
nores variation in outcomes that may be associated with factors cor- 
related with school poverty but not caused by the program itself. For 
instance, the introduction of the bonus policy coincided with an in- 
crease in the standard bonus for NBCTs in Washington and several 


Both counts 0.2102* 0.0814 

(0.0947) (0.0759) 
N 10,188 10,188 
CSRS FRL Count 0.0871 0.0161 

(0.0922) (0.0760) 
N 10,188 10,188 
CNS FRL Count 0.3092* 0.1731 

(0.1643) (0.1580) 
N 6,792 3,396 


Notes: We estimate the discontinuity in the forcing variable using the method of 
McCrary (2008a, b). The both counts measure is constructed using the full set of 
poverty measures considered by OSPI in the relevant year (see Table 1 for 
details). The CSRS count uses the FRL count from the CSRS data only. The CNS 
count uses the FRL count from the CNS data for 2008-2009 for eligibility during 
the 2008-2011 school years (column 1) and 2008-2009 school years (column 
2). The eligibility rule column uses the true eligibility rule for the relevant year, 
including historical data where appropriate, while the current enrollment 
column uses the current year's enrollment only. The standard errors are esti- 
mated using a school-level block bootstrap with 400 iterations. 

*p<0.10, 

** p =< 0.05, 

“Dp < 0.01. 


rounds of teacher layoffs during the recession; both of these factors 
likely have disparate effects in high and low poverty schools 
(Goldhaber, Strunk, Brown, & Knight, 2016). 

Previous research has suggested that schools react strategically to 
discontinuous eligibility rules. For instance, in an analysis of a large 
Northeastern school district, Matsudaira, Hosek, and Walsh (2012) 
found that schools clustered just above the eligibility threshold for 
school-wide Title I funding, and that this clustering followed changes in 
the eligibility threshold. Such manipulation of program eligibility po- 
tentially undermines the RDD. If schools can adjust their enrollment in 
order to qualify for the bonus program, then the treatment status of 
schools near the eligibility threshold may no longer be independent of a 
school's counterfactual outcomes. In the context of the CSB policy, 
schools with a greater baseline share of NBCTs might face greater in- 
centives to boost their FRL enrollment in order to qualify for the CSB, 
which would bias our estimates upward. 

As in prior research using schoolwide eligibility rules, we find that 
schools are significantly more likely to barely qualify for the CSB. The 
eligibility rule described in Table 1 is a complicated function of two 
series of FRL eligibility data. Let FRL“°** denote the CSRS poverty 
measure, FRL°’S denote the CNS poverty measure, FRL = max(FRL@RS, 
FRL“S) be the maximum of the two school FRL enrollment shares, and 
c be the school-specific threshold in Table 1. Then define 


Xst 
FRL, — 0.7 ift = 2008 
max(FRLs — Cst, FRLs¢—-1 — Cst—1) ift = 2009 
= max(FRLOSRS — Cst, FRLst—1 — Cst—1) ift = 2010 


max(FRLG°"* — cs, FRLGS4* — Cy—1, FRLs-2 — ¢st-2) ift = 2011 
FRLGSS — ¢,, ift > 2012 


(1) 


Schools are eligible for the CSB if and only if x,, = 0. We estimate 
the discontinuity in the density of this forcing variable in Table 3.° We 


8 We implement the density test using the Stata package DCdensity McCrary 
(2008a, b). Given the grouped nature of the data, we estimate standard errors 
using a clustered bootstrap on schools with 400 iterations. 
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find a statistically significant discontinuity in the density of 0.210, 
suggesting 21% more observations to the right of the eligibility 
threshold than expected. In column 2, we ignore the grandfathering 
provision and estimate the discontinuity in the maximum of the two 
poverty measures from the prior year only. The estimated discontinuity 
is 0.081 and not statistically significant. The fact that the discontinuity 
in the eligibility rule is larger indicates that it is not the same programs 
that just exceed the eligibility threshold in each year. 

The true eligibility rule uses two sources of data on FRL enrollment. 
One of these (CNS) is a survey completed by school administrators, 
while the other (CSRS) is based on administrative enrollment records. 
In the remaining rows, we show that the discontinuity in the forcing 
variable is largely attributable to the survey measure. In the first 
column, we reconstruct an alternative version of the forcing variable 
using only one of the sources of poverty data. This alternative measure 
uses the historical data in the same way, but omits the other poverty 
count. We then construct a contemporaneous measure using the only 
the prior year's poverty share. In the second row, we find little evidence 
of manipulation in the CSRS poverty count. The discontinuity in the 
forcing variable is 0.087 and not statistically significant; it is 0.016 
when looking at the contemporaneous measure only. The dis- 
continuities in the CNS measure are much larger, although they are 
used in fewer years and are therefore less precisely estimated. In the 
first column, we consider the CNS eligibility rule for 2008-2011. OSPI 
stopped using the contemporaneous CNS measure in 2010, but the 
grandfathering provisions ensured that historical CNS values were still 
used in determining eligibility through 2011. We estimate a dis- 
continuity in this variable of 0.309, which is statistically significant at 
the 10% level. In the second column, we show the discontinuity in the 
contemporaneous measure for 2008 and 2009, the two years in which it 
was used to determine eligibility. The discontinuity of 0.173 is smaller 
than in the first column and statistically insignificant, but much larger 
than the discontinuity in the CSRS measure (0.016). 

Given the evidence for manipulation in the CNS measure, we use 
only the CSRS count for the remainder of the analysis. We construct an 
analog of the eligibility rule using the CSRS data as in row 2, column 1 
of Table 3; that is, we replace FRL©®S for FRL in Eq. (1) above. We then 
implement the RDD using only this CSRS eligibility measure as a forcing 
variable. Because some schools qualify for the CSB without having a 
CSRS poverty count that exceeds the relevant threshold, this describes a 


fuzzy regression discontinuity. Using the forcing variable x“°"°, we 
estimate 

Yor = SCSBu + f xe") + es 

CSBy = YUOCESRS > 0) + gXSS®5) + ny, (2) 


using school-year aggregates of the teacher data and school-grade-year 
aggregates of the student data for 2008-2013. The one exception is the 
teacher mobility outcome, which should be related to the next school 
year's eligibility. In this case, we replace x°°** and CSB with their leads. 
We estimate fO by local linear regression with a triangular kernel and 
weighting by the number of observations in each school-year cell.? We 
estimate Eq. (2) using the optimal bandwidths for discontinuities in 
each of the outcome measures suggested by Imbens and 
Kalyanaraman (2012) for the grouped data, which are generally about 
0.35-0.40. In practice, we implement the regression discontinuity by 
weighted two-stage least squares. In addition to the baseline specifi- 
cation in Eq. (2), we estimate two variations of the RDD. First, because 
we pool data across several years and because the eligibility rules 
changed over time, the difference in treatment duration between just- 


° In the baseline models, we do not include any additional variables (such as 
grade level of year effects). Although the policy rule differs by school level and 
year, their inclusion should only affect the results if there are differences in the 
likelihood that schools are just-eligible or just-ineligible across grades and 
years. Regardless, their inclusion results in nearly identical point estimates. 
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eligible and just-ineligible schools varied by year. Between 2008 and 
2010, exceeding the eligibility threshold in one year guaranteed at least 
two years of CSB eligibility. We therefore instrument the number of 
years of cumulative eligibility, rather than current eligibility status, 
with the indicator for whether the school exceeds the threshold poverty 
level. Second, to exploit the panel nature of the data, we estimate 
models that include a number of school covariates, including all of the 
outcome measures from the year prior to the implementation of the CSB 
program. These include school grade level, school racial composition, 
current FRL enrollment share, log enrollment, 2007 mean student 
achievement in math and reading, and mean 2007 teacher NBCT status, 
experience, advanced degree possession, and turnover. 

The RDD provides an estimate of a local average treatment effect 
(LATE) of CSB eligibility for schools near the eligibility threshold. The 
effect of the bonus policy on teacher staffing may differ between 
schools near the eligibility threshold and those with higher poverty for 
two primary reasons. First, the probability of future bonus eligibility is 
lowest for schools that are just eligible in the current year. Relative to 
schools farther from the eligibility threshold, the value of job offers to 
NBCTs and earning professional certification should be less in just-eli- 
gible schools. For instance, the discounted value of a job offer to an 
NBCT from a just-eligible school exceeds that of a just-ineligible school 
by $7,250 between 2008 and 2010, by $8,601 in 2011, and by $5,000 
in 2012 to 2013.'° This value may be significantly larger in schools 
with substantially higher poverty shares: beginning in 2012, over a ten 
year horizon, the discounted value of an offer in a school guaranteed to 
receive the bonus exceeds an offer to a just-ineligible school by 
$21,881. By the same token, there are weaker incentives for certifica- 
tion in just-eligible schools. Moreover, because teachers only receive 
60% of the value of the bonus during their first year of certification, the 
financial value of certification is smaller than the value of a job offer to 
a previously certified teacher. The effects of the bonus may be further 
moderated by the district poverty shares. Teachers may take into ac- 
count the number of other nearby schools that offer the bonus when 
deciding whether to earn certification. To the extent that schools near 
the threshold are in higher income school districts than those with 
higher poverty levels, they may have fewer nearby schools that are 
eligible for the CSB. On the other hand, the value of nonpecuniary 
amenities may be highest near the eligibility threshold. Teachers 
therefore face a likely tradeoff between greater certainty surrounding 
bonus eligibility and other attributes of jobs in high poverty schools. 
The relationship between the effectiveness of the bonus policy and 
student poverty, and hence the relationship between the LATE we es- 
timate and the average effect of the policy on treated schools, is thus 
ambiguous. We return to the external validity of our findings in 
Sections 4.2 and 4.3. 


3.3. Testing the identifying assumptions of the RDD 


We briefly describe the relationship between the forcing variable 
and school eligibility in Fig. 2. We first plot the density of the forcing 
variable and its relationship with CSB eligibility. The discontinuity at 
the eligibility threshold, which corresponds to the estimate in Table 3, 
is 0.087 and not statistically significant. The forcing variable also pre- 
dicts eligibility for the bonus (right panel). Because eligibility is based 
on the maximum of the CSRS and CNS measures, every school to the 
right and some portion of schools to the left of the threshold receive the 
bonus. The first stage estimate of the discontinuity in eligibility status is 
0.845 with a t-statistic of 42.6. However, the just-eligible schools in- 
clude some whose eligibility is driven by their poverty counts 
in prior years. Thus, on average, those schools just eligible for the CSB 
have been eligible for about 1.5 years longer than those just 


1° We assume a discount rate of 3% and that the probability of future elig- 
ibility for schools just missing eligibility in the current year is 0.5. 
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Fig. 2. Density and Treatment Discontinuities at the Eligibility Threshold. Notes: In the left panel, we estimate the discontinuity in the forcing variable using the 
method of McCrary, 2008a, b). The scatter plot indicates the empirical density for bins of width of 0.005. We estimate a discontinuity of 0.087 (s.e. = 0.092). The 
standard error is estimated using a school-level block bootstrap with 400 iterations. In the right panel, we estimate the probability that a school is eligible for the 
bonus. The scatter plot indicates mean eligibility for cells of width 0.005 weighted by the number of teachers in each school. The line plot is the local linear 
conditional expectation using a triangular kernel, teacher observation weights, and a bandwidth of 0.34 (the Imbens and Kalyanaraman (2012) optimal bandwidth 


for the NBCT outcome). 


Table 4 
Test of covariate balance near eligibility threshold. 
Male 0.1249 Free- or reduced-price 0.4658 
lunch 
(0.2160) (0.7125) 
Asian/Pacific Islander 1.7868** Student attrition (math) 0.4416 
(0.7280) (0.5346) 
Black 2.1471**" — Student attrition — 0.0149 
(reading) 
(0.7355) (0.4739) 
Hispanic 0.1735 Student mobility 0.3876 
(1.2897) (1.5943) 
Gifted —0.0172 Predicted math — 0.0093 
achievement 
(0.9301) (0.0115) 
Limited English proficient 0.2710 Predicted reading — 0.0124 
achievement 
(0.6187) (0.0105) 
Special education 0.3287 
(0.2993) 
0.1249 
N 19,958 
Test of joint significance (p- 0.2117 


value) 


Notes: Student characteristics are the contemporaneous values of student de- 
mographic and program participation variables. Discontinuities are estimated 
by local linear regression with a bandwidth of 0.35. Standard errors clustered 
by school are in parentheses. 
*p < 0.10, 

* p< 0.05, 

** py < 0.01. 


ineligible.'’ 

We next turn to an analysis of discontinuities in pre-treatment 
outcomes and student characteristics at the eligibility threshold. In 
Table 4, we first investigate whether there are differences in the com- 
position in school enrollment at the eligibility threshold. Schools just to 
the right of the eligibility threshold have statistically significantly 
higher enrollments of African American and Asian students. None of the 
other discontinuities is statistically significant. Using the school-level 
data, we predict average achievement using the remaining student 
characteristics included in the table. The estimated discontinuities in 


11 Visual inspection of Fig. 2 suggests that the local linear regressions fit the 
data poorly for schools in the bin just to the left of the discontinuity. We revisit 
this issue in Section 5 and show that the results are not particularly sensitive to 
using a narrower bandwidth for the first stage regression. 


math and reading achievement are both about —0.01 and not statisti- 
cally significant. An additional concern is that school choice may be 
related to eligibility for the CSB program. Staffing improvements at 
eligible schools may lead to improvements in student retention and 
differentially affect the composition of test takers in eligible schools 
near the threshold. The attrition and mobility results in Table 4 suggest 
this is unlikely to be a problem in our study. We uncover little evidence 
of selective attrition. Point estimates in math and reading are 0.5 and 
—0.02 percentage points, respectively, and are not statistically sig- 
nificant. Similarly, the point estimate on student mobility is 0.3 per- 
centage points and not statistically significant. Overall, we fail to reject 
the null hypothesis that the discontinuities in student characteristics at 
the eligibility threshold are jointly zero. 

In Table 5 we test for differences in pre-treatment outcomes using 
teacher and student data between 2003 and 2007. We estimate these 
models using the 2008-2013 forcing variable and eligibility data and 
the 2003-2007 outcomes. The coefficients therefore describe the dis- 
continuities in baseline outcomes at the actual eligibility thresholds. 
Just-eligible schools tend to have fewer NBCTs prior to the adoption of 
the CSB program. The discontinuity for the year before implementation 
of the CSB, 2007, is — 0.4 percentage points and statistically significant. 
The remaining staffing outcomes are also generally worse on the right 
side of the eligibility threshold. The full set of discontinuities in Panel A 
are not jointly statistically significant over the full pre-treatment 
period, but they are statistically significant in 2007, the last year before 
the introduction of the bonus program.'* In Panel B, we test for dis- 
continuities in the student achievement outcomes. Before 2006, Wa- 
shington only tested in grades 4, 7, and 10. We therefore use the annual 
testing data in the two years before the implementation of the CSB for 
which end-of-grade tests are available for the same grades used in the 
main analysis. The estimated discontinuities are small and not statis- 
tically significant. 

The analysis of pre-treatment outcomes and student characteristics 
around the eligibility threshold may cast some doubt on the validity of 


12 The Institute of Education Sciences (2017) recommends examining the 
discontinuities in the standardized outcomes and including adjustments for any 
baseline measure with a discontinuity larger than 0.05 standard deviations. 
They further consider any discontinuity greater than 0.25 standard deviations 
reason for significant concern about the integrity of the research design. None 
of the discontinuities in the baseline measures in Table 5 is larger than 0.25 
standard deviations. Following this guidance, we show that the results are ro- 
bust to the inclusion of the baseline covariates, some of which have dis- 
continuities greater than 0.05 standard deviations. 
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Table 5 
Discontinuities in pre-treatment outcomes. 
2003 2004 2005 2006 2007 
Panel A. Staffing outcomes 
NBCT — 0.0003 — 0.0008 —0.0019* —0.0020 — 0.0043 
(0.0006) (0.0008) (0.0010) (0.0016) (0.0022) 
NBCT — 0.0000 — 0.0011 —0.0015 0.0008 —0.0024* 
Applicant 
(0.0009) (0.0014) (0.0017) (0.0013) (0.0013) 
Experience — 0.4373 — 0.2124 — 0.2371 — 0.1738 — 0.2087 
(0.1907) (0.1875) (0.1939) (0.1989) (0.2066) 
Advanced —0.0241 —0.0192 —0.0172* = —0.0172* — 0.0133 
degree 
(0.0097) (0.0093) (0.0093) (0.0086) (0.0083) 
Salary — 228.81 111.94 108.31 105.92 156.94 
(285.78) (284.66) (283.92) (297.25) (348.56) 
Turnover 0.0023 0.0038 — 0.0072 0.0128 0.0015 
(0.0059) (0.0060) (0.0113) (0.0062) (0.0061) 
N 6,514 6,708 6,753 6,643 6,877 
Test of joint 0.0610 0.1961 0.1068 0.1241 0.0473 
signifi- 
cance (p- 
value) 
Panel B. Student Outcomes 
Math test — 0.0012 — 0.0107 
(0.0176) (0.0178) 
Reading test 0.0020 — 0.0052 
(0.0173) (0.0176) 
N 18,913 19,169 
Test of joint 0.9596 0.7778 
signifi- 
cance (p- 
value) 


Notes: Pre-treatment outcomes give 2003-2007 values (by column) of key 
student and teacher variables for values of the running variable between 2008 
and 2013. Discontinuities are estimated by local linear regression with a 
bandwidth of 0.34. Standard errors clustered by school are in parentheses. 

* p<0.10, 

*“* p< 0.05, 

Dp < 0.01. 


the regression discontinuity design. In the analyses that follow, we take 
some steps to mitigate any potential bias from non-random sorting. In 
particular, we estimate models that include controls for all of the pre- 
treatment outcomes so that identification comes from changes in 
staffing and student achievement during the CSB period. We also show 
that the results are not sensitive to excluding schools very close to the 
eligibility threshold. Nonetheless, we argue that the patterns observed 
in Tables 4 and 5 indicate that our estimates of staffing effects likely 
provide a lower bound on the true treatment effect. Eligible schools 
tend to have worse pre-treatment staffing outcomes conditional on 
school poverty, which suggests that they are likely to have worse 
staffing outcomes during the CSB period as well. Although our esti- 
mates may be upwardly biased if schools that expect to hire more 
NBCTs in the future are able to manipulate their FRL enrollment to 
become eligible, the estimates in Table 5 do not suggest this is the case. 
There is no apparent upward trend in the discontinuities in the pro- 
portion of NBCTs at the eligibility threshold between 2003 and 2007. In 
fact, the point estimates are slightly more negative in 2007 than in 
earlier years. Furthermore, teachers in just-eligible schools are actually 
less likely to have applied for certification during 2007 and we do not 
observe any effects on any non-incentivized staffing outcomes (such as 
experience or educational attainment). 


4. Results 
4.1. Certification bonuses, school staffing, and student achievement 


We begin our presentation of the results with graphical evidence 
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from the regression discontinuity designs. The basic empirical results 
are summarized in Fig. 3. In the first panel, we show a discontinuity in 
the share of NBCTs of about 1.7 percentage points at the eligibility 
threshold. In the following panels, we show that there are also dis- 
continuities in the proportion of non-NBCTs earning certification, the 
proportion of NBCTs among newly hired teachers, and the turnover rate 
among incumbent NBCTs. Each year, non-NBCTs in just-eligible schools 
are 0.6 percentage points more likely to earn certification. Among 
newly hired teachers, those in just-eligible schools are about 0.9% more 
likely to be NBCTs. Finally, NBCTs in schools that are just eligible for 
the bonus in the upcoming school year are 2.7 percentage points less 
likely to leave their schools at the end of the year. 

We present more formal estimates for the NBCT staffing outcome in 
Table 6 that account for differences in eligibility at the threshold. We 
include the fuzzy regression discontinuity estimates in the first two 
columns, but focus our discussion on the results that use eligibility to 
instrument for cumulative treatment in the second two columns given 
the likely dynamic effects of the bonus program. Recall that changes to 
the grandfathering provision affect the length of eligibility guaranteed 
to schools that exceed the threshold. The IV estimators scale the re- 
duced-form treatment effects underlying the estimation of the simple 
RD by the discontinuity in the length of treatment at the eligibility 
threshold rather than by the difference in the probability of current 
treatment. These models therefore provide an estimate of the effect of 
an additional year of CSB eligibility. Our baseline estimates suggest that 
eligibility increased certification by about 1.2 percentage points per 
year of eligibility. In the final column, we include pre-treatment out- 
comes as additional controls. As expected given the lower 2007 certi- 
fication levels at the eligibility threshold, the inclusion of pre-treatment 
outcomes and other controls increases the estimated effect to about 1.7 
percentage points per year of eligibility. Among schools that are just 
ineligible for the bonus, about 6% of teachers are NBCTs, so this re- 
presents an increase of about 20-28% per year of eligibility. Put an- 
other way, the mean number of teachers in our sample is about 29.5, 
which suggests an overall increase of about 0.3 NBCTs per year. 

As we discussed above, the increase in the proportion of NBCTs 
could operate through any of three mechanisms: the bonus may in- 
crease the arrival rate of new NBCTs, it may increase the rate at which 
incumbent teachers earn certification, or it may reduce the departure 
rate of certified teachers. We separately consider each of these potential 
mechanisms in Table 7 by estimating regression discontinuity models 
using the relevant samples of teachers for each outcome. It is important 
to note that the RDD estimates effects on changes in the school average 
certification and mobility rates. If eligibility for the bonus affects the 
composition of a school's potential hires, then these effects are not 
necessarily the same as those for individual teachers. For instance, a 
school's eligibility may attract teachers who would be more likely to 
apply for certification even in the absence of the bonus. Increases in the 
certification rate or decreases in the turnover rate could therefore re- 
present compositional changes in a school's teaching staff caused by 
eligibility for the bonus rather than effects on the certification or 
turnover propensities of individual teachers. We therefore view these 
results as explorations of the mechanisms underlying the staffing 
changes rather than estimates of the effects of incentives on individual 
teacher behavior. Although the former are relevant to our context be- 
cause the policy targets whole schools, these effects may not generalize 
to the effects of system-wide retention incentives for NBCTs. In parti- 
cular, the effects we estimate likely overstate the effects of such state- 
wide policies. 

In Table 7, we find evidence that the CSB program improved out- 
comes through all three mechanisms. Because these outcomes influence 
the rate of increase of NBCTs, we focus on the current eligibility in- 
dicator rather than cumulative eligibility. In the first column, we esti- 
mate the effect of eligibility for the CSB program on the certification 
rates for previously uncertified teachers. The certification rates in 
treated schools are about 0.7 percentage points higher than in 
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Fig. 3. Regression Discontinuity Estimates of Staffing Effects. Notes: Scatter plot indicates mean eligibility for cells of 0.005 weighted by the number of teachers in 
each school. Line plot indicates local linear conditional expectation using a triangular kernel and weighting by the number of teachers in each school. For panels 2-4, 
the samples use only the relevant samples of teachers: teachers without professional certification (newly certified NBCTs), newly hired teachers (newly hired NBCTs), 
and NBCTs (NBCT turnover) with weights based on the size of the sample at the school level. The bandwidths are estimated using the procedure of Imbens and 
Kalyanaraman (2012). The bandwidths for each outcome are as follows: NBCT (0.34), newly certified NBCTs (0.45), newly hired NBCT (0.40), and NBCT turnover 


(0.40). 


Table 6 
Effect of the challenging schools bonus on proportion of board certificated 
teachers. 


Current Eligibility Cumulative Eligibility 

(1) (2) (3) (4) 
NBCT 0.0202*** 0.0280°** 0.0115 0.0164*** 

(0.0068) (0.0054) (0.0039) (0.0032) 
N 6,877 6,789 6,877 6,789 
Covariates N Y N a 


Notes: The regression discontinuity models are estimated by two-stage least 
squares with triangular kernel and observation weights. The bandwidth of 0.34 
is estimated using the procedure of Imbens and Kalyanaraman (2012). Cov- 
ariates include 2007 school values of the proportion of NBCTs, proportion of 
NBCT applicants, average teacher experience, average teacher educational at- 
tainment, average teacher turnover, math and reading achievement, student 
race/ethnicity, student FRL participation, and school year and school level ef- 
fects. Standard errors clustered by school are in parentheses. 
* p< 0.10, **p < 0.05, **p < 0.01. 


untreated schools, which represents an increase in the certification rate 
of about 42% compared to just-ineligible schools. In column 2, we re- 
strict the sample to newly hired teachers and estimate that the CSB 
program increased the proportion of newly hired teachers who were 
NBCTs by about 1.0 to 1.2 percentage points, or by about 38%. Finally, 
in column 4, we restrict the sample to current NBCTs and estimate the 
likelihood that a teacher leaves her school when it is eligible for the CSB 
in the following school year. We find a reduction of 3.2 to 4.2 per- 
centage points, depending on the specification, which corresponds to 
approximately 31-41% lower turnover rates among NBCTs. During the 
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Table 7 
Effects on certification rates, hiring, and retention. 
Outcome Sample (1) (2) 
NBPTS Certification Non-NBCTs 0.0072 0.0074 
(0.0023) (0.0022) 
N 8,409 8,300 
NBCT New Hires 0.0101* 0.0115** 
(0.0060) (0.0057) 
N 7,081 6,997 
Turnover NBCTs —0.0321* —0.0424*** 
(0.0142) (0.0137) 
N 4,041 3,970 
Covariates N Y 


Notes: The regression discontinuity models are estimated by two-stage least 
squares with triangular kernel and observation weights. The bandwidths are 
estimated using the procedure of Imbens and Kalyanaraman (2012). The sam- 
ples are as listed in the second column and weight are determined based on the 
number of teachers in each sample in each school and year. The bandwidths for 
each outcome are as follows: NBPTS application (0.38), newly certified NBCT 
(0.45), newly hired NBCT (0.40), and NBCT turnover (0.40). Covariates include 
2007 school values of the proportion of NBCTs, proportion of NBCT applicants, 
average teacher experience, average teacher educational attainment, average 
teacher turnover, math and reading achievement, student race/ethnicity, stu- 
dent FRL participation, and school year and school level effects. Standard errors 
clustered by school are in parentheses. 

*p<0.10, 

p< 0.05, 

“* p< 0.01. 
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period of our study, the average salary for NBCTs was $69,374 in just- 
ineligible schools. The bonus therefore represents an approximately 
7.2% increase in salary, which implies a turnover elasticity of between 
—4.3 and —5.7. The reduction in turnover is similar to the elasticity of 
about — 4 found in the Clotfelter et al. (2008) assessment of a retention 
bonus program in North Carolina. This is notably larger than those 
typically found in the literature on cross-district variation in teacher 
salaries (e.g., Hendricks, 2014), although these policies differ from pure 
salary differentials across districts in at least two important respects. 
First, differentiated compensation policies may affect within-district 
teacher mobility, which is likely less costly to teachers. Second, bonuses 
announced by the state may be more salient to teachers than differences 
in district salary schedules. 

Much of the existing literature on merit pay and certification bo- 
nuses assesses the impact on teacher retention. Our results suggest that 
hiring and the certification of incumbent teachers are empirically im- 
portant components of these policies. In order to provide a rough ac- 
counting of the contribution of these factors to the overall certification 
rate, we decompose the change in certification rates into portions at- 
tributable to each of the three mechanisms. In particular, we multiply 
the effects on the conditional rates of NBCT status given a teacher is 
new to the school, that a teacher becomes certified given that she is not 
already, and on turnover given that a teacher is an NBCT in Table 7 by 
the probability of teachers belonging to each of these groups. We use 
the regression discontinuity estimates of treatment effects for each the 
conditional probabilities and estimate the proportions of new and cer- 
tified teachers using similar local linear regressions. Recall that we 
estimate that the bonus increases the proportion of NBCTs by about 1.6 
percentage points per year. Of this increase, we estimate that newly 
hired NBCTs contribute about 0.2 percentage points, newly certified 
teachers contribute about 0.7 percentage points, and reductions in 
turnover contribute 0.3 percentage points. Given differences in the es- 
timation sample, the suggested increases in certified teachers do not 
perfectly match the estimates in Table 6, but the individual estimates do 
explain about 70% of the estimated annual increase in NBCTs.'* Con- 
sequently, much of the increase in certification appears to come from 
teachers already working in high poverty schools. 

The literature on NBCTs suggests that they improve student 
achievement by about 0.01 to 0.05 standard deviations relative to non- 
NBCTs (Cavalluzzo et al., 2015; Clotfelter et al., 2007,2010; Cowan & 
Goldhaber, 2016; Goldhaber & Anthony, 2007). Our staffing results 
therefore suggest that eligibility for the bonus policy increased average 
student achievement by about 0.0004-0.0005 standard deviations per 
year, although excluding the certification of incumbent teachers re- 
duces this amount significantly. Assuming a standard deviation of 
teacher effectiveness is equivalent to about 0.2 standard deviations in 
student achievement (Goldhaber, Liddle, Theobald, & Walch, 2012), 
these estimates correspond to an increase of about 0.2 to 0.3% of a 
standard deviation of teacher value added per year. In Fig. 4, we test 
the achievement effects directly using student test scores in grades 3 
through 8 and 10. There is little indication in the plots of any dis- 
continuity in test scores. We estimate discontinuities of 0.010 and 
—0.009 standard deviations in math and reading, respectively, al- 
though neither of these estimates is statistically significant. The formal 
analyses are consistent with the graphical evidence: the baseline RDD 
point estimates suggest that eligibility increases student achievement in 
math by about 0.006 standard deviations and reduces achievement by 
0.006 standard deviations in reading per year, but neither of these re- 
sults is statistically significant. The inclusion of covariates, including 
pre-treatment test scores, reduces the estimates in magnitude to 0.001 
and — 0.003. In both cases, the point estimates are near zero, although 


13 These figures use the estimates from the specifications with covariates. We 
find very similar results using models without covariates, although they explain 
about 90 percent of the estimated annual effect from Table 6. 
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they are not very precisely estimated; the 95% confidence intervals 
exclude annual increases of about 0.032 standard deviations in math 
and 0.013 standard deviations in reading. 

Taken together, our results suggest that the CSB policy increased the 
proportion of NBCTs in high poverty schools during the first five years 
of implementation, although the certification of incumbent teachers 
explains about half of the effect. We do not, however, find evidence of 
positive effects on student achievement. Based on previous findings on 
the productivity of NBCTs, any effects operating through direct in- 
structional effects are likely to be at most 0.3% of a standard deviation 
in teacher quality per year of eligibility. 


4.2. Dynamic effects of bonus policies 


In Section 4.1, we estimate a LATE of CSB eligibility using an RDD. 
The research design implicitly uses schools near the threshold to esti- 
mate the effect of an additional year of eligibility for the bonus. This 
effect may differ from the average effect on treated schools for two 
reasons. First, schools near the eligibility threshold face greater un- 
certainty about the likelihood of eligibility in future years, which re- 
duces the financial value of the bonus to teachers considering whether 
to apply for the bonus or transfer schools. Thus, all else equal, we 
should observe a weaker response in schools near the eligibility 
threshold. On the other hand, if teachers value working in schools with 
fewer low income students or other characteristics of schools that may 
be correlated with student income level, then schools near the elig- 
ibility threshold offer greater nonpecuniary value to teachers than 
schools with higher FRL eligibility shares. Thus, teachers should be 
more likely to switch to schools near the eligibility threshold than to 
schools with higher levels of student poverty. In either case, by com- 
paring eligible schools to ineligible schools near the threshold, the RD 
estimator may provide a misleading description of the overall effects of 
the policy. This may be especially problematic for predicting the effects 
of the policy in later years because the schools near the eligibility have 
likely had fewer years of total eligibility than schools with higher 
poverty levels. 

To test the representativeness of our RDD estimates, we compare the 
dynamic effects we estimate using the RD to instrument for cumulative 
eligibility with a difference-in-differences analysis of the introduction 
of the policy.'* Standard difference-in-differences models rely on in- 
eligible schools as a control to estimate counterfactual time trends for 
the treated schools. The underlying assumption is that treated schools 
would have followed similar trends as untreated schools in the absence 
of the CSB policy. The substantial differences in the poverty rates be- 
tween these groups makes this assumption problematic. We therefore 
conduct a difference-in-differences analysis using the sample of schools 
ever eligible for the bonus using data from 2003-2013: 


Yor = Xxe8 + NumYearsgd + a, + Ay + Ese. (3) 

We then allow for nonlinear effects by estimating an event study 
with separate coefficients for each year relative to the first year of 
eligibility for the CSB: 


14 An alternative to our difference-in-differences approach is to estimate a 
dynamic regression discontinuity model in the spirit of the one-step estimator of 
Cellini, Ferreira, and Rothstein (2010) that treats each lag of program eligibility 
as a separate RD. We do not pursue this approach here. In this context, the 
grandfathering provisions of the CSB policy and high re-enrollment rates for 
FRL programs ensure that there is high autocorrelation in both observed 
treatment and FRL enrollment share. Given that this method identifies lagged 
treatment effects by comparing schools with similar observed FRL histories but 
different patterns of treatment, estimates using this approach are very im- 
precise. Further, part of our interest in estimating alternative specifications is 
comparing the LATE estimated from an RDD to treatment effects estimated for a 
lager set of schools. 
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Fig. 4. Regression discontinuity estimates of student achievement effects. Notes: Scatter plot indicates mean eligibility for cells of 0.005 weighted by the number of 
students in each school-grade. Line plot indicates local linear conditional expectation using a triangular kernel and weighting by the number of students in each 
school-grade cell. The bandwidths are estimated using the procedure of Imbens and Kalyanaraman (2012) (math achievement, 0.41; reading achievement, 0.35). 
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Fig. 5. Event study of bonus introduction and school staffing. Notes: Event 
study estimates based on sample with FRL within 10 points of the eligibility 
thresholds. Estimates are relative to the first year a school is eligible. Covariates 
include student race/ethnicity, student FRL participation, and school year by 
school level effects. Standard errors clustered at the school level. 


Yoo = XB + Di_g FirstYeaty4.; + ts + Ay + €st- (4) 

The identification of the dynamic effects therefore comes from 
variation in the timing of initial eligibility among ever-eligible 
schools. !° Using the specification in Eq. (3), we estimate that an addi- 
tional year of eligibility increases the proportion of NBCTs by 0.0065 
(standard error = 0.0019). This is about half the estimated effects of 
0.0115 to 0.0164 from the RDD. To explore this discrepancy further, we 
plot coefficients from the event study specification in Fig. 5. The plot 
shows stable NBCT staffing levels before a school's initial eligibility 
followed by an increase immediately after the introduction of the 
bonus. The largest increase occurs in the second year after a school 
becomes eligible, although we find increases through the first five years 
of eligibility. Five years after a school's first eligibility, we estimate that 
the proportion of NBCTs has increased by 3.5 percentage points; this 
falls to 2.7 percentage points in the sixth year. The average effects are 
thus less than an extrapolation of the annual results from Table 6, 
which would indicate an increase of about 5.8-8.2 percentage points 
over the first five years. Although the graphical evidence in Fig. 3 
suggests the effects may have been largest for schools near the 


15 As with the analysis of the RDD above, we uncover little evidence of a 
detectible effect on student achievement. The results (available from the au- 
thors on request) are similar in magnitude and not statistically significant. 
There is also little consistent evidence of an effect from comparisons of mean 
test scores before and after the introduction of the policy. 
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eligibility threshold, the difference appears to result from the relatively 
higher weight put on earlier years of eligibility in the RDD. We estimate 
that schools just to the right of the eligibility threshold have had 1.94 
years of cumulative eligibility on average and the results in Fig. 5 
suggest that these years provide the largest effects on NBCT staffing 
levels. Even the effects shown in Fig. 5, however, represent an increase 
of about 45-60% in the number of NBCTs in high poverty schools over 
the first five to six years of eligibility. 


4.3. Spillover effects of teacher bonus policies 


The targeted bonuses increase pay for NBCTs, but only in certain 
schools. The program design may therefore negatively affect staffing in 
schools nearby eligible schools. Such effects would operate through the 
same channels as the positive effects on CSB staffing. For instance, some 
of the newly hired NBCTs in bonus-eligible schools may have left their 
previous schools in search of the bonus. But even if bonuses do not 
induce teachers to leave their schools, they may change the destination 
of mobile NBCTs. The policy may also lead teachers who are likely to 
become future NBCTs to sort into eligible schools. Each of these pos- 
sibilities would reduce the number of NBCTs in ineligible schools near 
those eligible for the CSB. 

To assess spillover effects, we estimate a difference-in-differences 
design similar to that in the prior section. As the main treatment vari- 
able, we use the proportion of teachers in other district schools that are 
eligible for the bonus. We construct this measure separately for each 
school level (elementary, middle, and high school) to account for the 
fact that bonus eligibility guidelines differ by school grade level and 
teacher credentials are often limited to a particular grade level. We also 
exclude a school's own status from the measure so that it is identified 
solely from eligibility in other district schools. As in the above analysis, 
we exclude high income districts that never have eligible schools. We 
then estimate 


Yyat = XyatB + CSByjatd, + CSB_Concentrationjg:d2 + a + Aje + €sjats 
(5) 


where j indexes school level and d indexes district.'° As with the ana- 
lysis in Section 4.2, this design relies on changes in school staffing 
outcomes following the introduction of the bonus policy in nearby 
schools. In some models, we additionally include district-by-year ef- 
fects, so that the concentration effect is identified based on changes in 
the number of eligible schools across different grade levels in the same 


16 Recall that mobility is defined as whether a teacher returns in the fol- 
lowing school year and that we use a lead of the CSB policy indicator to match 
end-of-year retention decisions with the job opportunities in the upcoming 
school year. 
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Table 8 
Spillover effects of bonus eligibility. 
q@) Q) (3) 4 
NBCT New hire NBCT NB certification 
turnover 
Panel A. District difference-in-differences estimates 
CSB 0.0243 0.0079 — 0.0307 0.0095 
(0.0047) (0.0034) (0.0194) (0.0010) 
CSB Concentration — 0.0223" — 0.0098 — 0.0081 —0.0065** 
(0.0085) (0.0072) (0.0265) (0.0025) 
N 8,600 8,132 2,996 8,600 
Panel B. District-by-year fixed effects estimates 
CSB 0.0262 0.0111 — 0.0260 0.0110 
(0.0051) (0.0041) (0.0206) (0.0013) 
CSB Concentration — 0.0132 0.0055 — 0.0208 — 0.0019 
(0.0095) (0.0087) (0.0414) (0.0039) 
N 8,569 8,081 2,838 8,569 
Panel C. Interaction effects (District DID) 
CSB 0.0288 0.0166 — 0.0370 0.0130 
(0.0068) (0.0059) (0.0276) (0.0020) 
CSB Concentration —0.0171* 0.0013 — 0.0140 — 0.0025 
(0.0094) (0.0094) (0.0327) (0.0030) 
CSB x CSB —0.0121 —0.0244* 0.0159 —0.0094** 
concentration 
(0.0118) (0.0126) (0.0441) (0.0045) 
N 8,600 8,132 2,996 8,600 
Panel D. Interaction effects (District-by-Year Effects) 
CSB 0.0202 0.0126* — 0.0288 0.0119 
(0.0072) (0.0073) (0.0317) (0.0023) 
CSB concentration —0.0186** 0.0070 — 0.0229 — 0.0012 
(0.0086) (0.0095) (0.0460) (0.0040) 
CSB x CSB 0.0181 — 0.0047 0.0078 — 0.0025 
concentration 
(0.0166) (0.0158) (0.0671) (0.0063) 
N 8,569 8,081 2,838 8,569 


Notes: Difference-in-differences estimates of CSB and CSB concentration effects 
estimated with observation weights. CSB concentration indicates the proportion 
of teachers in other district schools working in CSB-eligible schools. Covariates 
include school demographics and enrollment interacted with school level and 
level-by-year fixed effects. Models in Panel B additionally include district-by- 
year fixed effects. Models in Panels C and D additionally add interactions be- 
tween a school's CSB status and the CSB concentration measure. Standard errors 
clustered by district are in parentheses. 

*p<0.10, 

* p< 0.05, 

“= p< 0.01. 


district. This approach provides some guarantee that observed changes 
are not a consequence of district initiatives, such as additional stipends, 
release time for candidates, or loans for application fees, which may be 
associated with changes in the number of eligible schools. 

The results for bonus eligibility, in Table 8, largely mirror the results 
from the RDD and prior DID models. Although our focus is on esti- 
mating changes in the flow of NBCTs to eligible and nearby schools, we 
do find that teachers in eligible schools are about 2.4 to 2.6 percentage 
points more likely to be NBCTs, which matches the RD estimates. Si- 
milarly, the DID estimates indicate that eligibility increases the rate at 
which NBCTs are hired and certified by about 1 percentage point each 
and reduces NBCT turnover by about 3 percentage points. On the other 
hand, we find mixed evidence of spillover effects on other district 
schools. In Panel A, we estimate that the marginal effect of additional 
CSB concentration on total NBCT staffing in other district schools is 
— 0.02, which is statistically significant. However, the effect diminishes 
to —0.01, and is not significant, when we include district-by-year fixed 
effects in Panel B. In Panels C and D, we allow the effect of CSB con- 
centration to differ between CSB-eligible and CSB-ineligible schools and 
find that any CSB effect appears to be concentrated among ineligible 
schools. The main effect for CSB concentration, — 0.02, is significant 
and negative in Panel D with district-year fixed effects. The combined 
effect for CSB-eligible schools is near zero and statistically insignificant. 
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Thus, ineligible schools in districts with higher concentrations of CSB 
eligible schools experienced reductions in their overall NBCT staffing 
levels. The average concentration among ineligible schools in districts 
with at least one CSB school was 0.38, which suggests a total reduction 
of about 0.2 NBCTs per ineligible school. 

We find inconsistent evidence on specific mechanisms by which the 
bonus reduces the proportion of NBCTs in other district schools. 
Although the coefficients on the rate of newly hired NBCTs and new 
certifications are in the expected direction in the difference-in-differ- 
ences models in Panel A, only the coefficient on new certification is 
statistically significant. We estimate that having nearby eligible schools 
actually reduces NBCT turnover, although the estimate is not statisti- 
cally significant. In Panel B, with district-by-year effects, none of the 
coefficients on district concentration is significant. Consistent with 
these findings, we do not estimate significant effects of CSB con- 
centration among ineligible schools in Panels C or D. Taken together, 
the results do not provide clear evidence of effects of targeted bonuses 
on staffing in nearby schools, although they do appear to affect staffing 
in ineligible schools. This finding may be dependent on the design of 
the CSB policy. The policy targets a credential that any teacher could 
hypothetically earn, and we find that much of the effect of the bonus on 
a school's workforce is driven by increases in certification rates among 
incumbent teachers. Thus, the effect of the bonus need not be entirely 
offset by reductions in NBCTs in other schools. In particular, the policy 
does not appear to induce mobility among NBCTs in neighboring 
schools. 

Readers may be concerned that the potential for negative spillover 
effects on ineligible district schools biased upward our RD estimates 
above. In particular, if the bonus induces teachers to move from eligible 
to ineligible schools, the changes in NBCT staffing at the eligibility 
threshold may over count the increase in NBCTs among eligible schools. 
Although we do not find evidence that the bonus affects NBCT turnover 
in ineligible schools, we do find that having more eligible schools in a 
district reduces NBCT staffing for ineligible schools. Nonetheless, the 
left limit on the concentration measure among ineligible schools at the 
eligibility threshold is only 0.06, suggesting an upper bound on the 
potential spillover bias of 0.001.’” 


5. Additional robustness checks 


In Table 9, we conduct a number of sensitivity tests of the RDD. In 
the first three columns, we test the specification of the relationship 
between the forcing variable and NBCT staffing outcomes by im- 
plementing the RDD using different bandwidths. Each entry displays 
the coefficient on the cumulative years of CSB eligibility. In columns (1) 
and (2), we use half and twice of the Imbens and Kalyanaraman (2012) 
optimal bandwidth. The estimated effects are generally similar to main 
results. Using half the optimal bandwidth, the NBCT effect is not sta- 
tistically significant without covariates, but the point estimate for an 
additional year of eligibility (0.009) is similar in magnitude to the es- 
timate in Table 6. The point estimate with covariates (0.014) is statis- 
tically significant. At twice the optimal bandwidth, the point estimates 
(0.010 and 0.015) are statistically significant and again similar to the 
results in Table 6.'* Finally, we test the sensitivity of our results to the 
specification of the first-stage regression. In the baseline regressions, we 
scale each of the discontinuities in outcomes by the discontinuity in the 


17 We compute this by multiplying the limit on CSB concentration by the 
effect estimated in Panel D for ineligible schools. Note that the bias will be less 
than this since not all schools at the threshold are ineligible. 

18 as an additional test of the specification of the local linear regressions, 
Imbens and Lemieux (2008) recommend testing for discontinuities in the out- 
come variables at the median points of the forcing variable above and below the 
eligibility threshold. Neither of the estimated discontinuities is statistically 
significant at any conventional level (results available from the authors upon 
request). 
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Table 9 
Sensitivity Tests for the Regression Discontinuity Design. 
q) Q) (3) (4) G) (6) 
Baseline RDD 0.0088 0.0102 0.0092* 0.0142 0.0125 0.0138*** 
(0.0062) (0.0030) (0.0051) (0.0053) (0.0039) (0.0041) 
N 3,662 10,119 4,686 6,877 6,767 6,415 
RDD with covariates 0.0144** 0.0147 0.0153 0.0206 0.0175* 0.0185 
(0.0052) (0.0025) (0.0042) (0.0043) (0.0032) (0.0035) 
N 3,613 10,000 4,621 6,789 6,679 6,330 
Bandwidth 0.5x (0.17) 2x (0.68) FS (0.22) Separate >0.5pt >2pt 


Notes: The regression discontinuity models are estimated by two-stage least squares with triangular kernel and observation weights. The treatment variable in each 
model is the cumulative number of years of CSB eligibility. The models in columns (1) and (2) use half and twice the Imbens and Kalyanaraman (2012) optimal 
bandwidth, respectively. The models in column (3) use the optimal bandwidth for the treatment variable rather than the outcome variable. The estimates in column 
(4) estimate the first and second stage using optimal bandwidths for each stage (0.22 for the first stage and 0.34 for the second stage). The estimates in columns (5) 
and (6) are from “donut” regression discontinuity models that omit observations within 0.5 and 2 percentage points of the eligibility threshold, respectively. 
Covariates include 2007 school values of the proportion of NBCTs, proportion of NBCT applicants, average teacher experience, average teacher educational at- 
tainment, average teacher turnover, math and reading achievement, student race/ethnicity, student FRL participation, and school year and school level effects. 
Standard errors clustered by school are in parentheses. Standard errors clustered by school are in parentheses. Standard errors in column (4) estimated by school- 
clustered bootstrap with 399 iterations. 


* p<0.10, 

“Dp < 0.05, 

“ p< 0.01. 
number of years of eligibility and use the optimal bandwidth for the the positive correlation between assessment scores and teacher value 
outcome variable. In column (3), we use the optimal bandwidth for the added indicated by Cantrell, Fullerton, Kane, and Staiger (2008) and 
treatment variable instead. The point estimate without covariates Cowan and Goldhaber (2016), then a signaling argument would suggest 
(0.009) is only statistically significant at the 10% level, although the that newly certified teachers were on average less effective than those 
estimate with covariates (0.015) is statistically significant. Finally, in earning the certificate in the absence of additional bonuses. That is, we 
column (4) we use the optimal bandwidth for each stage of the IV re- would expect that teachers on the margin of seeking certification were 
gression separately and estimate the effect of an additional year of less effective on average than those who would otherwise have ob- 
eligibility as the ratio of the two discontinuities. The plot of eligibility tained the credential. If this were the case, then the bonus policy may 
and school FRL in Fig. 2 indicates that we may overestimate the change have partially undermined the usefulness of professional certification as 
in the probability of treatment at the discontinuity, which would tend a tool for attracting effective teachers to high poverty schools. This may 
to depress our estimates of the CSB effect. Although the point estimates provide another explanation for the null findings of the effects of the 
(0.014 and 0.021) are slightly larger than the baseline estimates, the policy on student achievement. 
magnitude is similar and both pairs of coefficients are statistically sig- The student achievement data provides a direct measure of teacher 
nificant. productivity. We can therefore test whether teachers who earned cer- 

As we discuss above, the main empirical findings are robust to the tification in schools eligible for the CSB were less effective than those 

inclusion of pre-treatment outcome variables. However, the dis- who earned certification in ineligible schools. To do so, we use a sample 
continuities in lagged outcomes and student characteristics suggest of students matched to classroom teachers in elementary and middle 
caution about interpreting the results as causal. In the remaining col- school grades.'° We first compare the effectiveness of teachers earning 
umns, we provide further tests of whether manipulation in the forcing certification while teaching in a school eligible for the bonus to other 
variable may be driving our results. Our primary concern is that schools certified teachers. We estimate 
may be able to perturb their FRL enrollment counts in order to become 
just eligible for the CSB. If this kind of manipulation is associated with Yijse = XiseB + NBCTj5, + NBCTy X CertCSBj52 + ase + eyse (6) 
potential outcomes, then the RDD is no longer valid. We therefore 
follow Barreca, Guldie, Lindo, and Waddell (2011) and re-estimate the where X is a vector of student characteristics, including lagged test 
models after omitting schools closest to the eligibility threshold. This scores, NBCT indicates that a teacher earns certification in our sample 
approach should remove potential manipulators, but relies on an ex- period, CertCSB indicates that the teacher earned certification while her 
trapolation based on schools farther from the discontinuity. We omit school was eligible for the CSB, and a is a school-by-year fixed effect. 
schools within 0.5 percentage points of the eligibility threshold in This model is mainly descriptive in that it makes no attempt to control 
columns (5) schools within 2 percentage points in column (6). In both for differences in the effectiveness of teachers earning certification at 
cases, the estimates are similar to the main results in Table 6. Consistent different types of schools nor adjust for any differences in selectivity 
with the negative sorting on baseline outcomes apparent in Table 5, the across certification tests or time. To provide a better sense of whether 
point estimates are actually somewhat larger when omitting schools eligibility influences the pool of certified teachers, we estimate a dif- 
near the threshold. It does not appear that manipulation in the forcing ference-in-differences type specification of Eq. (6) that additionally 
variable significantly influences our results. adds an indicator for earning certification in a school that is ever eli- 


gible for the CSB and either an indicator for earning certification after 
2008 (the first year of the CSB program) or certification test-by-cohort 
fixed effects. 

The results of these regressions, in Table 10, exhibit no evidence 
that the certification bonus reduced the effectiveness of newly certified 
teachers. The coefficients on NBCT status in the first row of each panel 


6. Certification bonuses and the signaling value of credentials 


One potential difference between the CSB and other targeted com- 
pensation policies is that mid-career teachers can relatively easily earn 
certification. The incentive to earn certification could plausibly lead the 
CSB policy to affect the underlying relationship between student 
achievement and the targeted credential. If we assume that certification 19 See Cowan and Goldhaber (2016) for a description of the sample used in 
is more costly for less effective teachers, which seems reasonable given this analysis. 
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Table 10 
The effectiveness of teachers certified with bonus eligibility. 
q@) (2) G) (4) G) 
Panel A. Math teachers 
NBCT 0.0332*** 0.0340 0.0156 0.0200 
(0.0076) (0.0067) (0.0124) (0.0115) 
NBCT, eligible 0.0458** 0.0149 0.0285 0.0093 0.0266 
school 
(0.0176) (0.0144) (0.0250) (0.0210) (0.0213) 
NBCT, ever- 0.0106 — 0.0009 — 0.0130 
eligible school 
(0.0221) (0.0188) (0.0189) 
NBCT, post-2008 0.0245 0.0202 
(0.0154) (0.0136) 
N 1,312,566 1,312,566 1,312,566 1,312,566 1,312,566 
Panel B. Reading teachers 
NBCT 0.0250°*** 0.0174 0.0294 0.0107 
(0.0050) (0.0047) (0.0089) (0.0088) 
NBCT, eligible 0.0031 0.0051 0.0094 0.0067 0.0072 
school 
(0.0121) (0.0114) (0.0174) (0.0172) (0.0174) 
NBCT, ever- — 0.0052 — 0.0054 — 0.0064 
eligible school 
(0.0150) (0.0149) (0.0149) 
NBCT, post-2008 — 0.0056 0.0104 
(0.0107) (0.0101) 
N 1,234,910 1,234,910 1,234,910 1,234,910 1,234,910 
School-by-year FE N bg N Y Y 
Certification test- N N N N Y 


by-cohort FE 


Notes: Estimates are from student-level data over the period 2007-2013. The 
sample is described in Cowan and Goldhaber (2016). All models include con- 
trols for cubic polynomials in lagged achievement, student race/ethnicity, 
subsidized lunch status, participation in special education, bilingual education, 
and gifted education services, and the classroom means of these variables. 
“NBCT” indicates the teacher is board certified, “NBCT, eligible school” in- 
dicates the teacher is board certified and earned certification in a school cur- 
rently eligible for the CSB, “NBCT, ever-eligible school” indicates that the 
teacher earned certification in a school that is ever eligible for the CSB, and 
“NBCT, post-2008” indicates that the teacher earned certification in 2008 or 
after. Certification test-by-cohort fixed effects are defined as unique combina- 
tions of the certificate type and year. Standard errors clustered by school are in 
parentheses. 
*p < 0.10, 

p< 0.05, 

“ep < 0.01. 


indicate that students of National Board certified teachers perform 
about 0.02-0.03 standard deviations higher on state assessments in 
math and reading, depending on the model. These results are consistent 
with prior research on NBCTs (Cantrell et al., 2008; Clotfelter et al., 
2007; Cowan & Goldhaber, 2016; Goldhaber & Anthony, 2007). We do 
not find that teachers certified in CSB-eligible schools are less effective 
than those earning certification either before the program began or in 
ineligible schools; however, the point estimates are positive and sta- 
tistically significant for math in models without school fixed effects. 
Further, it does not appear that eligibility for the CSB changes the ef- 
fectiveness of teachers earning certification. The difference-in-differ- 
ences models in columns (3) through (5) find no reduction in the ef- 
fectiveness of NBCTs in eligible schools (indicated by the coefficient on 
NBCT, eligible school). 


7. Conclusion 


We study the introduction of an incentive for National Board cer- 
tified teachers to work in high poverty schools and find that the bonus 
increased the proportion of teachers with the professional certificates. 
Depending on the method of analysis, we estimate that after five years 
of eligibility for the bonus, the percentage of board certified teachers 
would have increased by about four to eight percentage points. As in 
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other studies of differentiated compensation for high needs schools, we 
find a reduction in the turnover rates for affected teachers. However, 
the bonus program also appears to have changed the characteristics of 
newly hired teachers. Although the findings of Protik et al. (2015) 
suggest that transfer bonus policies may not induce large numbers of 
teachers to switch schools, our results indicate that they may affect the 
composition of a school's applicant pool. About half of the increase is 
explained by teachers in eligible schools becoming certified. Although 
eligibility for the bonus increased the likelihood that incumbent tea- 
chers apply for certification, we find little evidence that the introduc- 
tion of the bonus policy diluted the signaling value of the certificate. 
Teachers credentialed in high poverty schools following the introduc- 
tion of the bonus are at least as effective as those credentialed before- 
hand. 

Importantly, however, the increase in the number of NBCTs does not 
appear to have led to detectible improvements in student achievement, 
a finding which is consistent with the magnitude of staffing changes we 
observe as well as the prior evidence on the instructional effects of 
NBCTs. Nonetheless, it is possible that the CSB policy may have influ- 
enced student outcomes through other mechanisms. There is little di- 
rect evidence for NBCTs’ effects on non-tested student outcomes, but 
Gershenson (2016) and Jackson (2016) suggest that achievement gains 
may be poor predictors of other dimensions of teacher effectiveness. 
Instead, the strength of the relationship between student outcomes and 
observable teacher characteristics may be a limiting factor for differ- 
entiated compensation policies based on teacher credentials. Although 
several prior studies have found that NBCTs are more effective in the 
classroom, observable credentials explain little of the variation in tea- 
chers’ contributions to student achievement. The achievement findings 
from this study may therefore not generalize to policies that use direct 
performance measures, such as teacher evaluations, to target incentives 
(e.g., Adnot, Dee, Katz, & Wyckoff, 2017). 


Supplementary materials 


Supplementary material associated with this article can be found, in 
the online version, at doi:10.1016/j.econedurev.2018.06.010. 
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